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Abstract:Real life jobs need interdisciplinary skills and educations should be adjusted in a way that 
students obtain these skills in their education lives. Designing interdisciplinary online courses seems to 
be unavoidable in educational institutions to have the technological and innovational catch-up in the 
21" century. This paper includes a discussion about the importance of the assessment methods and 
presents BWG assessment method as an option. It is measuring students’ success in three factors with 
BWG Assessment: Behavioral Success, Work Success and Group Success. A teaching method, as an 
alternative to STEM and STEAM approaches, combining the expertise of Science, Technology, 
Engineering, Arts and Math with specialized, student-centered Unique projects called USTEAM was 
delivered to lycée students’ for 9 months. To find students’ attitude towards this new method USTEAM 
Attitude test is used. Descriptive Statistics, Compare Means and General Linear Models were used in 
IBM SPSS Statistics 25 for analyzing the data obtained from BWG pretest and BWG posttest marks of 
the Standard Online Courses and the USTEAM Online Courses. Results was awarding parts of the 
study and success was more than what was expected. Although some students were aggressive at the 
beginning toward being a member of USTEAM groups, they were more motivated by assuring that they 
didn’t have any competitors and they were expected to produce their Unique projects regardless of 
what others do! For motivation theorists, the results from this study provide evidence that 
interdisciplinary cultural influences in courses can be expressed in diverse educational settings. For 
educators, this study underlines the importance of cooperative and creative USTEAM activities in 
online courses. 
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I. INTRODUCTION 


1.1 What is STEM or STEAM? 


Students perceptions and attitudes plays an important role on the effectiveness of the teaching 
methods. However the first version of the combined disciplines method STEM was often associated 
with negative perceptions and learning difficulties [1, 2,5], which was the main reason of the 
curriculum designers increased efforts for modifying STEM integrating arts and encouraging creative 
solutions with a newer approach STEAM [3]. This new approach claimed to help students to transfer 
enthusiasm from artistic work to science to support individual self-efficacy and thus, to close the 
‘creativity gap’ and may provide additional channels to learn abstract mathematical problem-solving 
tasks [6,7]. However all of these exclude the personal differences and needs of students. Students in 
schools are coming from all folks of life bringing their own expertise of life each. Some of this 
background experiences are very good examples for students developments, however some were very 
bad and specially the students with bad experiences needed more attention. Today, in governmental 
schools students coming from rich families and poor families are classmates. So, how can a poor 
student buy the necessary research equipment’s necessary for the STEAM? It would not be fair to 
expect same experiences, performance and facilities from every student. Each student should be 


136 


evaluated his own Unique capacities and should be counselled to work on his capabilities. On the 
other hand there are lots of students coming from a divorced family, from an unhappy family or even 
from a family in which parents are both dead. There are also students with lots of personal differences 
such as dyslexic or hyperactivity in classrooms. In addition to all these, day by day classes get more 
multicultural. These days in most of lycée classes we have students from Turkey, Cyprus, Russia, 
England, Australia and Azerbaijan. None of the mentioned methods until today are considering these 
differences among students, hence for all these reasons another newer approach considering these 
differences, considering Uniqueness of students is needed! 


1.2 Why USTEAM? 


USTEAM (Unique Science, Technology, Engineering, the Arts and Mathematics) is an 
educational approach that uses different disciplines as access points for guiding student questions, 
dialogues, problem solving strategies, imagination, critical thinking and producing unique solutions all 
valid in different cases. In this method more responsibility is on students and they are forced to think 
more, take their own risks and engage in experiential learning collaboration and become more aware 
of the culture and the community they live. Instead of blank-minded computer youth who are sitting in 
front of computers and playing more than 10 hours computer games, we need innovators, educators, 
leaders, and learners of the 21st century. That is why we have to change our educational methods and 
make sure that we help them to be the one that they dream! 


1.3 Assessment in Online Courses 


Assessment and evaluation is a topic which is mostly omitted for years by educators, 
researchers and curriculum designers. However, it is one of the most important part of the education 
process that helps us teachers to deliver more effective courses. Today there are not many researches 
on how we can better asses and evaluate students process in online courses. Mostly the assessment 
methods of the students are the multiple choice tests like the current assessment method of the most 
universities. Does crossing a’s, b’s or c’s need a special talent? Or is it really the wanted outcome of 
our courses? Does crossing or clicking letters mean that we have reached our aims of the course? 
Although it was known that the assessing and the integration of technology into the classrooms is one 
of the biggest challenges of educational decision makers and school administrators face [4], not much 
is done in most of the offline or online educations to overcome this challenge. 


1.4 BWG Assessment Method 


How we understand when our students acquired the knowledge? It doesn’t seem completely 
true to assess students from one-type multiple choice questions. In 21* century more than that is 
needed. For example when we really learn something it may be observed from our behaviours to the 
topic as a result of the influence and impact of the new knowledge has had on us. Of course, the 
extend of the change of behaviour is directly proportional with the ingrained negative behaviour and 
the habits of the person. So, the behavioural change can be assessed in time and is one of the good 
outcomes of a well designed courses. When observing the change in the behaviour, one should not 
omit the factor or importance of the gained knowledge and skills. Most history courses in schools are 
memorized by the students and after a few years they are forgotten. So, why we are keeping our 
students mind busy with the things that they will forget in time? Why we are not trying to help 
students to keep their own history in their minds for a lifetime and help them to represent our culture 
as a grown up businessman? Students start a race once they start in their school life and continue until 
they graduate from the school. They learn how to attain in a race in school life and they always try to 
be more successful from their friends in the real life. However, in real life we need more than racers! 
We need people who are knowledgeable, who can think, who can produce and who can share their 
knowledge with others for the sake of the community. We have to teach students how to collaborate, 
how to work together in schools so that we can expect these when they grow up and become our 
workers. That is why assessment and evaluation of their collaborative work is also necessary. This 
study offers BWG Assessment method where Students’ knowledge and skills are measured in 3 
different ways: “Their change in Behavior (B)”’, “Their Work Marks (W)”, “and Average Group 
Marks (G)”. 
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1.5 Purpose 


Primary aim of this research study is to find if there is a meaningful difference between the 
students pretest BWG and posttest BWG marks in the Standard online courses and in the USTEAM 
Online courses. Secondary aim of the study is to find the students’ attitude towards USTEAM courses. 


Il. METHOD 


2.1 Population 


240 students, ages between 15-17 in Cyprus have attained to the research study. 120 students 
were registered to the USTEAM Online Courses and 120 students attained to the Standard Online 
Courses. 


2.2 Instruments and Process of Data Collection 


Both groups actively participated to the online courses for 9 months, in total 72 hours. Course 
materials were uploaded and shared students via Bloggers and Social Media Group, face-to-face chats 
were Students directed by teachers to choose their own unique projects and prepared their own videos 
and uploaded to the bloggers. Figurel shows some of the students videos that they had prepared and 
uploaded online. They all had different projects according to their interests and had to report each 
week on the course blog their studies during the week by uploading a writing document or a video. 
Standard Online courses were following the standard teaching and learning methods such as teachers 
explaining a subject asking questions and students answering those. However, USTEAM online 
courses students were free to do their unique project in the guidance of the USTEAM council in the 
school (which contained 18 different branch teachers; Chemistry, Physics, Biology, Technology, 
Computer Engineer, Electronics, Turkish, English, French, German, Geography, History, Arts, 
Psychology, Philosophy, Economics, Music, Councilor). Students had to choose 4 different teachers 
each week, show their work and take their advice. Every Friday, students were presenting their work 
to their Technology teacher in the online class chats and sharing their activity videos with their 
friends. Although some students were shy at the beginning and were not very eager to attain to the 
class meetings and present their weekly report, they get used to it in time. Students getting used to 
presenting their own work are encouraged them to continue and do their best. 


Figure no.1. Students’ Online Videos 
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Teachers from different diciplines Science,Technology, Engineering, Arts and Mathematics 
collaboratively worked on the course curriculumn and prepared the USTEAM course materials for 
students. Descriptive statistics were used to analyze and to report the data gained from the 
questionnaire. For data analysis, SPSS 25.0 was used. Students’ knowledge was measured in different 
ways: “Their change in Behavior (B)”, “Their Work Marks (W)”, “and Average Group Marks (G)”. If 
the change was observed they got 1 and if the change was not observed they get 0. Every student Got a 
“behavioral success” mark each week, and the average of these (in 10 weeks) is accepted as B success 
mark for this study. This mark was given as | (if B is higher than or equal to 50) and 0 (if B is lower 
than 50). Similarly, each week, students was marked for their “work success”. W Marks were given 
between 0 until 100, according to the rubrics prepared for each course. At the end of 10 weeks average 
of these marks were taken and this formed the W success mark. This mark was given as | (if W is 
higher than or equal to 50) and O (if W is lower than 50). G Marks were also given between 0 until 100 
(if G is higher than or equal to 50) and 0 (if G is lower than 50). Average score of the marks are given 
according to the rubrics prepared for each course. At the end of 10 weeks average of these marks were 
taken and this formed the G success mark for this study. 

Attitude test was applied to the students at the end of 10 weeks. Any relation between students 
BWG successes and their attitude or perception was investigated. After having taken the experts 
suggestions; the questionnaire was divided into 3 factors: Behavioral Success Attitude (B success 
attitude), Work Success Attitude (W success attitude) and Group Success Attitude (W success 
attitude). In these questionnaire students, attitudes towards USTEAMSs resulting in their Behavioral 
Work Success and Group Success are measured. The scales of the attitude questionnaire have values 
changing from 1 (strongly disagree) to 5 (agree). Interviews were carried with the course teachers to 
get their opinion about their students behavioral changes. 


Il. RESULTS AND DISCUSSION 


As a result of 9 months USTEAM Online activities, Pre-BWG Marks and Post BWG Marks 
provided us interesting results. In 3.1 results of students attitude tests was presented 


3.1 Students Attitudes toward USTEAM 


Students attitudes towards USTEAM can be seen in the Table no.1. This table shows that 
students have positive attitude towards USTEAM. 


Strongly : ee Strongly 
Sex : Disagree | Agree nor | Agree 
Disagree : Agree 
Disagree 
Adaptable/Flexible nema ° 2 i Se i 
Male 3 3 15 33 12 
USTEAM Allows Female 0 0 2 30 22 
Collaboration 
and Cooperation Male 3 3 8 36 16 
USTEAM Should be in | Female 0 4 17 14 19 
school curriculum Male 3 6 11 27 19 
USTEAM Projects are | Female 0 0 29 21 
Enjoyable Male 3 3 6 39 15 
We do Feasible Studies | Female 0 0 7 32 15 
with USTEAM Male 3 3 18 28 14 
I feel Aggressive in Female 23 24 3 2 2 
USTEAM Projects Male 39 17 4 6 0 
USTEAM is Quite Female 29 15 6 2 2 
Frustrating Male 43 15 8 0 0 
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I feel Happy doing Female Z 6 1 19 26 


USTEAM projects Male 2 11 12 19 22 
Helps me to improve Female i UI 10 i Ml 
my _ skills. Male 19 9 19 9 10 
Improves my Female ) 11 17 9 12 
knowledge. Male 9 13 23 11 10 
USTEAM helps me Female 0 0 7 30 7 
improve my social 
skills. Male 3 5 12 33 “13 
USTEAM is just not Female 24 23 2 1 4 
forme Male 27 32 6 1 0 
Iam more eager to Female 1 7 6 39 1 
learn with USTEAM Male 1 13 16 33 3 


Provides opportunity to 


use ubiquitous learning | Female 2 12 12 7 11 
tools Male 2 15 14 25 10 
USTEAM makes me Female 0 6 13 19 16 
creative Male 2 17 28 11 
I can easily voice out Female 6 16 12 10 10 
options in USTEAM 

Groups Male 11 14 19 11 11 
I want to use USTEAM_ | Female 0 6 13 19 16 
in all my courses Male 2 8 17 28 11 


Table no.1. USTEAM Attitudes 


According to this results and the descriptive statistical analysis the following interpretations 
are made: % 57, 96 of them agreed “USTEAM" will be useful for them for developing social 
communication skills.” (M = 3, 53, SD = 1, 12). Students have positive attitudes for USTEAMs being 
used to improve communication skills. %70, 42 feel happy doing “USTEAM” works (M = 3, 93, SD = 
1, 07). %74, 58 believe that the more often that teachers use "USTEAM", the more they will enjoy the 
school (M = 3, 44, SD = 1, 34). % 87, 5 of them did not agree with the item about "USTEAM" being 
frustrating (VM = 1, 59), SD = 0, 86); and %90, 41 of them did not agree with the item "USTEAM" 
doing any good (M = 1, 69, SD = 0, 80). These are important; since if they do not think that it is 
frustrating, then they do not have a negative attitude to it. If they do not have a negative attitude, then 
they have no objections to the USTEAM’s rules such as; respecting others; behaving as a unique 
person to each member in their groups. 

% 40 of the students said that they know “USTEAM" will be useful for them for improving 
their organization skills (M =2, 97, SD = 1, 41). %42, 08 of the students said that they learn more 
when they are attaining a “USTEAM” course than other courses (M = 3, 27, SD = 1, 25). What is 
more, %64,58 of them study more for “USTEAM” courses than the other courses (M = 3, 52, SD = 0, 
90) and %56,25 of them wants to use "Unique" game in their courses when they become a teacher (M 
= 3, 45, SD = 1, 29). %56, 67 of "Unique" should be used in schools curriculum, (M = 3, 54, SD = 1, 
31) and %61/, 25 of them believe that more and more people will be using “USTEAM" (M = 3, 44, SD 
= 1, 34). Only a low percentage of students (%15) think that is a "waste of time". We do not need 
games to learn things (M = 1, 72, SD = 1, 07). As well as all these, %60,83 of the students think 
"USTEAM" helps them to develop "Feasible Studies" (M = 3, 46, SD = 1, 28). Work success attitude 
of students were positive (M > 2, 5). 

%43, 33 of the students think that in "Unique" groups, it is easier to voice out my opinions 
than speak them face to their instructor (M = 3, 13, SD = 1, 38). %68, 33 of them claims that they have 
the opportunity to use ubiquitous learning tools (M = 3, 58, SD = 0, 85) and a high percentage of them 
(%63, 75) think that they help them to be Adoptable (M = 3, 60, SD = 1, 23). %74, 17 of them think 
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that it is easier to collaborate and cooperate in "USTEAM" classes than the traditional classes (M = 
3,75, SD = 1, 28). %87, 5 of them feel aggressive toward being a member of "USTEAM" groups (M = 
1, 65, SD =0, 93). 


USTEAM ATTITUDE 


Strong Disagree Disagree Neither Agree nor Agree Strongly Angree 
Disagree 


eeeeFemales == Males 


Figure no.2. USTEAM Attitude of Male and Female Students 


It can be seen in the figure no.2 above that especially male students have a more positive 
attitude than female students and they think that the USTEAM courses are more enjoyable than the 
traditional ones. 


3.2 BWG Marks of Standard Online Courses and USTEAM Online Courses 


A paired-samples t-test was conducted to compare Pre-BWG Marks which were obtained 
from the tests delivered to students before courses and the Post-BWG Marks which were given at the 
end of the courses. There was a significant difference in the Pre-BWG Marks (M=1.09, SD=0.988) 
and Post-BWG Marks (M=1.80, SD=1.14); t(239)=-23.18, p=0.00. However, these tests were not 
enough to compare the USTEAM Online courses and the Standard Online courses. To find if there is 
significant difference between USTEAM Online group and Standard Online group controlling 
PreBWG Univariate test was delivered (see Table no.2). 


95% Confidence 
Interval of the 
Std. Std. Error | Difference 
Mean _ | Deviation | Mean Lower Upper 
Pair 1 | Post-BWG -.708 | .473 031 -.769 -.648 - 239 | .000 
Pre-BWG 23.175 


Table no.2. Pre-BWG & Post-BWG Test 


Means of USTEAM and Standard Online courses was calculated. Levene's Test of Equality of Error 
Variances, Type III Sum of Squares was analysed and it was found that there is a significant difference 
between the Post-BWG Marks of USTEAM Online groups and Post-BWG Marks of Standard Online 
groups controlling the effect of the Pre-BWG achievements, F (1,237)=69.87, p=0.00 (see Table no.3) 
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Type HI Sum 
Source of Squares df Mean Square |F Sig. | Partial Eta Squared 
Corrected Model | 198.846 2 99.423 686.649 | .000 | .853 
Intercept 854 1 854 5.895 .016 | .024 
Pre-BWG 16.842 1 16.842 116.316 | .000 | .329 
Group 10.116 1 10.116 69.866 000 | .228 
Error 34.316 237 145 
Total 517.000 240 
Corrected Total | 233.163 239 


Table no.3. Univariance Analysis of Groups 


IV. CONCLUSIONS AND RECOMMENDATION 


There is a huge research gap in the education area about the correct assessment and evaluation 


methods. Since our students come with different facilities and personal differences we have to adjust 
our educational system according to the changing community. USTEAM offers more than STEAM 
and STEM and this study shows that it has positive effects on students BWG Marks. Univariate results 
show that there is a significant difference between Post BWG Test Marks of Online USTEAM Classes 
and Standard Online Classes, taking Pre-BWT Test Marks as covariate. Attitude test results show that 
students with USTEAM method define themselves to be more creative and more attentive and more 


happy. 


What are more male students has more positive attitude than female students towards 


USTEAM. As a result of the current research study, it is recommended to educational administrators 
to include USTEAM in their curriculums and to encourage more interdisciplinary studies. 
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